Image processing apparatus, image processing method, and computer readable medium and computer data signal for processing image

ABSTRACT

An image processing apparatus includes: a unit producing a histogram in a color space from an input image; a unit extracting a plurality of peak colors having a local maximum frequency of appearance of a color value in the histogram; a unit determining whether to unify the extracted plurality of peak colors, based on a feature amount of the extracted plurality of peak colors, and selecting a peak color as a representative color when the extracted plurality of peak colors are unified, the feature amount being a lightness difference among the extracted plurality of peak colors and directions of vectors in the color space, the vectors each showing a shortest distance from one of the extracted plurality of peak colors to a line segment connecting between a reference color and a dark color; and a unit replacing a color of each pixel in the input image by the representative color.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 USC §119 fromJapanese Patent Application Nos. 2006-216278 and 2007-99880, filed Aug.8, 2006 and Apr. 5, 2007, respectively.

BACKGROUND

(i) Technical Field

The present invention relates to an image processing apparatus, an imageprocessing method, and a computer readable medium and computer datasignal for processing image.

(ii) Related Art

In the related art, it is a practice to implement a color-limitingprocess to reduce the number of colors in use on an image. Thecolor-limiting process allows for reducing the various noises andirregularities contained in the image, e.g. printing irregularities onthe original and scanning noises caused in reading out where the imageis a read one out of the original by means of an image reader, andhand-written irregularities where hand-written ones are included.Meanwhile, in compressing the image, the process can reduce compressionnoises while improving the ratio of compression.

The color-limiting process is to be realized by selecting some number ofrepresentative colors out of the colors used in the image and thenreplacing the colors of the image into the representative colors.

SUMMARY

According to one aspect of the present invention, there is provided animage processing apparatus comprising: a histogram producing unit thatproduces a histogram in a color space from an input image; a peak-colorextracting unit that extracts a plurality of peak colors having a localmaximum frequency of appearance of a color value in the histogram; arepresentative-color selecting unit that determines whether to unify theextracted plurality of peak colors, based on a feature amount of theextracted plurality of peak colors, and that selects a peak color as arepresentative color among the extracted plurality of peak colors whenthe plurality of peak colors are unified, the feature amount comprisinga lightness difference among the extracted plurality of peak colors anddirections of vectors in the color space, the vectors each showing ashortest distance from one of the extracted plurality of peak colors toa line segment connecting between a reference color and a dark color;and a replacing unit that replaces a color of each pixel in the inputimage by the representative color.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary Embodiments of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a block diagram showing a first exemplary embodiment of theinvention;

FIGS. 2A to 2C are figures to explain an exemplary example of ahistogram producing section;

FIGS. 3A to 3C are figures to explain an exemplary example of apeak-color extracting section;

FIG. 4 is a block diagram showing an exemplary example of arepresentative-color selecting section;

FIG. 5 is a flowchart showing an exemplary example of operation of arepresentative-color extracting section 42;

FIG. 6 is an explanatory figure of an exemplary example of a peak-colorunifying process at the representative-color extracting section 42;

FIG. 7 is an explanatory figure of an exemplary example of a peak-colorunifying-processed at the representative-color extracting section 42;

FIG. 8 is a flowchart showing an exemplary example of additionalextraction of a representative color;

FIGS. 9A and 9B are explanatory figures showing an exemplary example ofadditional extraction of a representative color;

FIGS. 10A and 10B are explanatory figures showing an exemplary exampleof a representative color to additionally extract;

FIGS. 11A and 11B are explanatory figures showing another exemplaryexample of unifying peak colors at the representative-color extractingsection 42;

FIGS. 12A to 12C are figures to explain an exemplary example of apixel-value replacing section 14;

FIG. 13 is a block diagram showing a second exemplary embodiment of theinvention;

FIGS. 14A and 14B show an exemplary concept of a filter coefficient fora filtering section 15; and

FIG. 15 is an explanatory figure of an exemplary example of a computerprogram, a storage medium storing such a computer program and acomputer, where an image processing apparatus of the invention isrealized in its function by means of a computer program.

DETAILED DESCRIPTION

Exemplary embodiments of the invention will be discussed with referenceto the accompanying drawings.

FIG. 1 is a block diagram showing a first exemplary embodiment of thepresent invention. In FIG. 1, 11 is a histogram producing section, 12 isa peak-color extracting section, 13 is a representative-color selectingsection and 14 is a pixel-value replacing section. The histogramproducing section 11 counts a color value in each pixel in an inputimage and prepares a histogram in a color space.

The peak-color extracting section 12 extracts a peak color maximal(local maximal) in frequency, among the histogram prepared by thehistogram producing section 11. Incidentally, when extracting the peakcolor, also specified is another peak color that is a reference color(e.g., a paper color).

The representative-color selecting section 13 determines whether or notto unify a plurality of peak colors, based on a feature amount of theplurality of peak colors extracted by the peak-color extracting section12. The representative-color selecting section 13 also selects a peakcolor to be left, as a representative color among the plurality of peakcolors to unify. Whether or not to unify a peak color can be determinedbased on, as the feature amount, a lightness difference among theplurality of peak colors and directions of vectors in the color space,the vectors representing a shortest distance from each peak color to aline segment connecting between the reference color and a dark color(e.g., a black color). Meanwhile, when the plurality of peak colors areunified, it is possible to determine which one of the peak colors isallowed to be left upon unifying, based on at least one of a colorsaturation, a hue, a lightness, a frequency and a differential value offrequency, of the peak color.

Incidentally, a peak color, specified at the peak-color extractingsection 12, can be used as the reference color (a paper color) thatprovides an end point of the line segment for defining the vector.Meanwhile, as for the dark color, it is possible to determine whether ornot a peak color lowest in lightness of the peak colors can be regardedas black, and to set the peak color as a black color in the case thepeak color lowest in lightness can be regarded as black or to set anarbitrary black color in the case the peak color lowest lightness cannotbe regarded as black.

The pixel-value replacing section 14 replaces pixel-based colors of theinput image by the representative colors selected by therepresentative-color selecting section 13. Due to this, the input imageis limited in its used colors to the representative colors, thuseffecting a process of color limitation.

Explanation is made continuously on the above configuration. FIG. 2A isan explanatory figure of an exemplary example of the histogram producingsection. In the figure, 2 is a pixel-value quantizing section and 22 isa histogram calculating section. The histogram, in a color space, may bedetermined by counting the number of pixels on a color value-by-colorvalue basis. However, because the data will be huge in amount, shownhere is an example to count the number of pixels by gathering the colorvalues in a certain range.

In the configuration example shown in FIG. 2A, the histogram producingsection 11 has a pixel value quantizing section 21 and a histogramcalculating section 22. The pixel-value quantizing section 21 quantizesa color value in each pixel. For example, by rounding down thelower-order several bits as to each color component, quantization can bemade on the color value. FIG. 2B shows an exemplary example to count thenumber of pixels as to the blocks of 3×3×3 within the entire RGB colorspace by dividing color components (R, G, B) into three parts in the RGBspace. Naturally, although this example illustrates to make a countingbased on rough blocks divided less in the number for the sake ofillustration, the actual accuracy of quantization can be establisheddesirably. For example, in an image read by a scanner, there is apossible occurrence of delicate color dissimilarity on a domain of theimage despite the domain is uniform in color on the original image. Inalso such a case, a merit is available that delicately dissimilaritiesof colors can be gathered together in an equal color value by quantizingthe pixel value at the on-pixel value quantizing section 2.

The histogram calculating section 22 counts the quantized color valueand prepares a three-dimensional histogram on a color space. FIG. 2Cshows an exemplary example of a histogram distribution in an RGB colorspace. By counting the pixel-based quantized color values, the resultingcount values (other than 0) in most cases come in a distribution in apart of the color space, as shown by the hatching. In the explanationfrom now on, there are shown only a partial space where distribution iswith other count values than 0.

FIGS. 3A to 3C are explanatory figures of an exemplary example of thepeak-color extracting section 12. In FIG. 3A, 31 is a secondarydifferential filter and 32 is a paper-color extracting section. As notedbefore, the peak-color extracting section extracts a peak color maximalin frequency, from the three-dimensional histogram prepared at thehistogram producing section 11. For a configuration for such a purpose,a secondary differential filter 31 is provided. By performing athree-dimensional processing of secondary differential filtering on thethree-dimensional histogram by means of the secondary differentialfilter 31, maximal values (peak colors) can be extracted out of thethree-dimensional histogram. For the secondary differential filter 31,techniques in the related art can be applied. Naturally, a peak colormay be extracted by another processing than that of the secondarydifferential filter, e.g. primary differential filtering. Otherwise, aplurality of peak-color extraction processes can be combined in use.

The peak colors, extracted by the secondary differential filter 31, aredenoted at black circles in FIG. 3B. The main colors used in the inputimage, the colors particularly different in hue from others, etc. areextracted as peak colors.

In this example, the peak-color extracting section 12 is provided with apaper-color extracting section 32 to extract a paper color. Thepaper-color extracting section 32 is to cope with also the case thebackground color is not white, e.g. image read out of an originaldocument such as a newspaper or a colored paper. The paper-colorextracting section 32 is allowed to specify a paper color by means of afunction having parameters of the secondary differential value andlightness obtained by the secondary differential filter 31. In manycases, the color of image background is high in frequency so that one ofthe peak colors obtained at the secondary differential filter 31 can beextracted as a paper color. Naturally, background-color extractionprocess may be performed separately. Alternatively, a predeterminedcolor can be taken as a paper color, to provide a configuration withoutproviding such a paper-color extracting section 32. The color, extractedas a paper color, is shown at a white circle in FIG. 3C.

FIG. 4 is a block diagram showing an example of the representative-colorselecting section 13. In FIG. 4, 41 is a black determining section, 42is a representative-color extracting section and 43 is alow-saturation-color excluding section. As noted before, therepresentative-color selecting section 13 selects a representativecolor, to replace the pixel of the input image, out of the peak colorsextracted by the peak-color extracting section 12. For a configurationfor that purpose, the example shown in FIG. 4 has a black determiningsection 41, a representative-color extracting section 42 and alow-saturation-color extracting section 43.

The black determining section 41 sets up a black color that is to beconnected through a line segment to the paper color at therepresentative-color extracting section 42. At first, determination ismade as to whether or not a peak color, lowest in lightness of the peakcolors extracted at the peak-color extracting section 12, can beregarded as black. For example, determination can be made under such acondition with a lightness being in a value or smaller. When the peakcolor can be regarded as black, the peak color is established as black.On the contrary, when the peak color cannot be regarded as black, ablack color is set up. For example, the black color may be set up for animage not using black.

The representative-color extracting section 42 extracts a representativecolor out of the peak colors extracted at the peak-color extractingsection, by use of the black color established at the black determiningsection 41. In extracting a representative color, a line segmentconnecting between the paper color and the black color is used fordetermining a vector representing a shortest distance between the linesegment and the peak color. Using a direction of the vector andlightness difference, similar colors are gathered together.Simultaneously, determination is made as to which color is to be left asa representative color out of the peak colors to unify. Detailed processis described later.

The low-saturation-color excluding section 43 excludes low-saturationcolors out of the extracted representative colors and to determine arepresentative color. Because a low-saturation color in most cases fallsunder gray and hence can be replaced, as a representative color, withblack. Meanwhile, low saturation colors exist in positions close to alightness axis or a pseudo lightness axis. Since hue in this regionchanges greatly in value, the representative-color extracting section 42possibly makes an erroneous extraction. Accordingly, representativecolors having low saturation are deleted here. Naturally, this processmay be omitted.

The representative-color extracting section 42 is further explained.FIG. 5 is a flowchart showing an operation example of therepresentative-color extracting section 42. At S51, a set of two peakcolors unprocessed is selected out of the peak colors extracted at thepeak-color extracting section 12.

At S52, determination is made as to whether or not to unify the two peakcolors. The determination, as to whether or not to unify the peakcolors, may be performed in the following way. FIG. 6 is anexemplification figure of a unifying process of peak colors at therepresentative-color extracting section 42. At first, in a color space,a line segment connects between a paper color and a black color. Thisline segment can be considered as a pseudo lightness axis though notalways coincident with the lightness axis because the paper color is notnecessarily white. Vectors are determined that are the shortest indistance from the line segment to the two peak colors under processing.In FIG. 6, those are shown as vectors V1, V2.

As for the relationship between the vectors V1 and V2, first determinedis an angle Δθ defined between the vectors V1 and V2. The angle Δθ isindicative of a difference amount in hue relative to the pseudolightness axis. Where the difference amount is small, the colors arehighly possibly similar to each other and hence are to be unitedtogether.

Meanwhile, a lightness difference ΔL* between the two peak colors isdetermine. Where the lightness difference is small, the colors arehighly possibly similar to each other and hence are to be unified.

From the angle Δθ and lightness difference ΔL* thus obtained,determination is made comprehensively as to whether or not to unify thetwo peak colors. Namely, where there is a great difference in lightnessdespite small is the angle Δθ indicative of a hue difference amount,those in many cases are to be recognized as different colors. Meanwhile,where there is a great difference in hue even if small in lightnessdifference ΔL*, those are naturally recognized as different colors. Forthis reason, when the both are small, the two peak colors can bedetermined to be unified. Specifically, the following is provided as anevaluation function:H=a×Δθ+b×ΔL* (a, b: constants).In the case the value H calculated is smaller than a threshold,determination can be made for unifying.

In the case determined not to unify at s52, the process proceeds to S54.When determined to unify, one of the two peak colors to unify is deletedwhile the other is left at S53, thus making a unifying substantially tothe left peak color. How to determine which one is to be left can use acolor saturation, a hue, a lightness, a frequency and a differentialvalue of frequency or the like of the peak color to unify. Specifically,the length |V1|, |V2| of the vector V1, V2 determined before can be usedfor the color saturation. Provided that the frequency (histogram countvalue) of the peak color is freq(V1) and freq(V2) while the differentialvalue of the frequency is (freq(V1) and (freq(v2)), G1 and G2 arecalculated by means of an evaluation function Gi given in the following:Gi=c×|Vi|+d×freq(Vi)+e×(freq(Vi))′where c, d and e are constants while i=1, 2. If G1>G2 is held whencomparing between G1 and G2 calculated, the peak color corresponding toG1 is determined to be left. If G1≦G2, the peak color corresponding toG2 is determined to be left. The other peak colors, than thosedetermined left, are deleted.

At S54, it is determined whether or not there is left a set of twounprocessed peak colors. If there is a left one, the process returns toS51 where the process is repeated for the unprocessed set. Afterprocessed on all the sets, the processing at the representative-colorextracting section 42 is ended.

FIG. 7 is an explanatory figure of an example of the peak colorunite-processed at the representative-color extracting section 42. Thesimilar peak colors, of the FIG. 6 peak colors other than those in paperand black colors, are unified into the peak colors shown at doublecircles in FIG. 7. The deleted peak colors are shown at broken-linedcircles.

The peak colors left in the unifying process are extracted asrepresentative colors and taken as outputs of from therepresentative-color extracting section 42. Of the extractedrepresentative colors, the representative colors having low saturationsare excluded by the low-saturation-color excluding section 43. Therepresentative colors left are provided as selected representativecolors by the representative-color selecting section 13. Incidentally,the paper and black colors are left as representative colors. In thismanner, the color arrangement in the input image can be correctlydetermined as representative colors.

In the foregoing representative-color extracting method, a peak colorwas extracted as a representative color which is different in thedirection of a vector indicative of a shortest distance between thepseudo lightness axis and each peak color to process or is distant inposition therefrom. Accordingly, the peak colors, existingintermediately on a vector corresponding to a representative color, areunified instead of being extracted. Those colors, in positions close tothe pseudo lightness axis, are also not extracted but excluded by thelow-saturation-color extracting section 43. Of among the peal colors notextracted but unified, there is possibly included a color to bedesirably extracted as a representative color. Now explanation is madeon a method of additionally extracting a representative color.

FIG. 8 is a flowchart showing an exemplary example of a process toadditionally extract a representative color. FIGS. 9A and 9B areexplanatory figures on a process example to additionally extract arepresentative color. At first, at S101 in FIG. 8, a region isestablished where to perform re-extraction. Such a region is establishedby setting up colors C1, C2. Of the peak colors existing close to a linesegment connecting between the colors C1, C2, if there is a color totake as a representative color, the relevant color is extracted. Whereto add a representative color close to the pseudo lightness axis, thecolors C1, C2 are respectively taken with a paper color and a black.Meanwhile, where to add as a representative color a color intermediatelyon a line segment indicative of a shortest distance between a pseudolightness axis and a representative color, the colors C1, C2 arerespectively taken with a projection point of the representative coloronto the pseudo lightness axis and the respective color.

At S102, a peak color existing close to the line segment connectingbetween the colors C1, C2 is taken as a color-being-noticed Ck. Forexample, it is possible to take a peak color Ck, existing in a distancerange from the line segment connecting between the colors C1, C2, as acolor-being-noticed Ck. There are cases that a plurality of peak colorsexist close to the line segment connecting between the colors C1 and C2.In such a case, steps S103 to S106, explained in the following, areexecuted by considering those as colors-being-noticed Ck.

At S103, determined is an angle α defined between a vector V (C1, Ck)and a vector V (Ck, C2). FIG. 9A illustrates the relationship betweenthe colors C1, C2, Ck, the vectors and the angle α. Thecolor-being-noticed Ck is a color in a position close to the linesegment connecting between the colors C1 and C2. In this case, bydetermining the angle α defined between the vector V (C1, Ck) and thevector V (Ck, C2), the angle α shown is determined. The angle αdecreases when the color-being-noticed Ck is closer to the line segmentC1-C2. Even where distant equally from the line segment, the angle α issmaller in a position intermediate between the colors C1 and C2. Forexample, if it exists besides the color C1, the vector V (C1, Ck) has anincreasing angle with the line segment, thus providing the angle αgreater. Likewise, if the color Ck is besides the color C2, the vector(Ck, C2) and the line segment give a greater angle, thus providing theangle α greater.

At step S104, a quadric differentiated value F2 of frequency isdetermined on the color-being-noticed Ck with respect to a direction ofthe line segment (direction of the vector (C1, C2)). As for a frequencydistribution, it is possible to use a three-dimensional histogramprepared at the histogram producing section 11. Meanwhile, the quadricdifferentiated value may be calculated by use of a filter, as shown inFIG. 9B. Incidentally, although the filter is practically threedimensional, it is shown here in a two dimensional form for the sake ofconvenience. Meanwhile, FIG. 9B shows an example of a coefficient matrixof a two-dimensional differential filter, at an angle of 45 degreescorresponding to the line segment C102 shown in FIG. 9A. Thecoefficients differ depending upon the direction of the line segment.Meanwhile, the filter is desirable in size. The quadratic differentiatedvalue F2 represents at what degree the color-being-noticed Ck, as a peakcolor, is used distinct as compared to the surrounding colors.

At S105, an evaluation value H2 is determined from the angle αdetermined at the S103 and the quadratic differentiated value F2determined at the S104. The evaluation value H2 is calculated by anevaluating function that outputs a greater value as the angle α issmaller and the quadratic differentiated value F2 is greater, forexample. Incidentally, the evaluation value H2 may be normalized at thefrequency value of the color-being-noticed Ck.

At S106, the estimated value H2, calculated at the S105, is comparedwith a threshold. When the estimated value H2 is greater than thethreshold, the color-being-noticed Ck is tentatively extracted as arepresentative color. Incidentally, where there are a plurality of peakcolors in positions close to the line segment C102 as in the foregoing,the steps of from S103 to S106 are carried out by taking the peak colorsas representative colors Ck. Due to this, the color-being-noticed Ck, tobe tentatively extracted as a representative color, is not limited toone in the number but possibly be extracted in plurality. Naturally,there are cases that nothing be extracted at a certain extracted valueH2.

At S107, in the case there are a plurality of colors-being-noticed Cktentatively extracted as representative colors at S106, those inpositions near to one another on a color space (e.g. those within apredetermined threshold) are grouped together. Of thecolors-being-noticed Ck belonging to the same group, the greatest one inthe estimated value H2 is additionally extracted as a representativecolor. Otherwise, by selecting desired two out of thecolors-being-noticed Ck belonging to the same group, the peak colorgreater in the estimated value H2 is left as a representative-colorcandidate when existing within a constant distance on the color spacewhile the other peak color is excluded from the representative-colorcandidate. The procedure may be repeated to extract the remainingrepresentative-color candidate as a representative color. In thismanner, additionally extracted is a color that was existent on the linesegment C1-C2 and to be extracted as a representative color.

Incidentally, the representative-color additional extraction shown inFIG. 8 may be performed plurality in number of times by changing thecolors C1, C2 to select at the S101. For example, it is possible toextract a gray representative color by selecting paper and black colorsas colors C1, C2, and to additionally extract representative colorsrespectively from the representative colors extracted at therepresentative-color extracting section 42 and the projection points ofthe representative colors onto the pseudo lightness axis. In this case,the representative-color additional extraction may be made on all therepresentative colors extracted at the representative-color extractingsection 42. This makes it possible to additionally extract arepresentative color even where there are a plurality of colors that areequal in hue but representative colors are not to be perfectly extractedby the representative-color extracting section 42. Meanwhile, inadditional extraction of a gray representative color, therepresentative-color extracting section 42 makes an accurate extractionof the low-saturation representative colors excluded by thelow-saturation-color excluding section 43, including those erroneouslydetected at the representative-color extracting section 42.

FIGS. 10A and 10B are explanatory figures showing an exemplary exampleof a representative color to additionally extract. FIG. 10A shows a casethat paper and black colors are selected respectively as colors C1, C2,showing an example that peak colors Ck1-Ck4 exist close to a linesegment. In the example, peak colors Ck1-Ck4 are respectively taken ascolors-being-noticed, to execute steps S103-S106. Extracted tentativelyare those whose evaluated values H2, corresponding to the peak colorsCk1-Ck4, are greater than a threshold. For example, the four are assumedtentatively extracted as representative colors. Because the four peakcolors are such mutually approximate colors as within a given distanceon a color space, the four peak colors are grouped together to extractthe greatest one in estimated value H2 as a representative color. Forexample, assuming that the peak color Ck1 has the greatest estimatedvalue H2, the peak color Ck1 is extracted as a representative colorwhile the other peak colors Ck2-Ck4 are not taken as a peak color.Otherwise, by selecting desired two out of the peak colors Ck1-Ck4, thepeak color having a greater estimated value H2 is left as arepresentative color when those are within a given distance on the colorspace while the other peak color is excluded from therepresentative-color candidate. The procedure is repeated to extract theremaining representative-color candidate (peak color Ck1, in this case)as a representative color. In this manner, the peak color Ck1 could beadditionally extracted as a representative color on the case that paperand black colors were respectively selected as colors C1, C2.

FIG. 10B shows a case that a color C2 is selected with a representativecolor extracted at the representative-color extracting section 42 and acolor C1 with a projection point of the representative color thereofonto a pseudo lightness axis. In this example, peak colors Ck5-Ck7 areshown existing in positions close to a line C1-C2. Because the peakcolors Ck5-Ck7 has a vector, indicative of a shortest distance to thepseudo lightness axis, that is in the same direction as a vectorindicative of a shortest distance of the representative color C2 to thepseudo lightness axis, those are to be united together by therepresentative-color extracting section 42.

In also this example, the peak colors Ck5-Ck7 are respectively taken ascolors-being-noticed, to execute the steps S103-S106. For example, it isassumed that the evaluated values H2 corresponding to the peak colorsCk1-Ck4 are greater than a threshold wherein all the three have beententatively extracted as representative colors. Because the three peakcolors are colors approximate one to another, the three peak colors aregrouped together to extract as a representative color the greatest onein estimated value H2. For example, assuming that the peak color Ck6 hasthe greatest estimated value H2, the peak color Ck6 is extracted as arepresentative color while the other peak colors Ck5, Ck7 are notselected as a representative color. Otherwise, by selecting desired twoout of the peak colors Ck5-Ck7, the peak color having a greaterestimated value H2 is left as a representative color when those arewithin a given distance on the color space while the other peak color isexcluded from the representative-color candidate. The procedure isrepeated to extract the remaining representative-color candidate (peakcolor Ck6, in this case) as a representative color. In this manner, thepeak color Ck1 could be additionally extracted as a representative coloron the basis of the representative colors extracted at therepresentative-color extracting section 42.

FIGS. 11A and 11B are explanatory figures showing another example tounite peak colors together at the representative-color extractingsection 42. In this example, a straight line passing through peak andpaper colors is considered to unify the peak colors existing close tothe straight line. In this case, out of the peak colors to unify,selected is a color distant as greater as possible from the paper color.By thus unifying the peak colors, the remaining peak color is onceextracted as a representative color. The representative colors thusextracted are shown at black circles in FIG. 11A.

Such extraction of a representative color has been carried out whereinextraction is not made as to the representative colors existing betweenthe color extracted as a representative color and a paper color.Therefore, by means of the method explained in FIG. 8, a paper color anda selected representative color are respectively selected as colors C1,C2, to additionally extract a representative color.

For example, in the example shown in FIG. 11B, some peak colors shown atdotted-lined circles and double circles exist together at between apaper color and representative colors (black circles). In such a case,if the peak colors shown at dotted-lined and double circles are unitedwith the representative colors shown at black circles, color differenceresults great. By additionally extracting a representative color fromthe peak colors existing between each representative color and the papercolor by means of the method shown in FIG. 8, the peak color shown atthe black circle, for example, is additionally extracted as arepresentative color. Due to this, besides a color distant as great aspossible from the paper color, a color intermediately existing can alsobe extracted as a representative color that is high in utilizationfrequency and distinct from other colors.

FIGS. 12A to 12C are explanatory figures of an exemplary example of thepixel-value replacing section 14. In FIG. 12A, 61 is aprojected-straight-line calculating section and 62 is a replacingsection. As noted before, in the pixel-value replacing section 14, thepixel-based color of the input image is replaced with any of therepresentative colors selected at the representative-color selectingsection 13. In order to perform the process, there is a need to select arepresentative color to replace out of the pixel-based colors of theinput image. Showing is made on one method to select such arepresentative color. Incidentally, in FIGS. 12B and 12C, white circledenotes a paper color, black circle a representative color, and whitecircle with “+” a color on a pixel where the input image is present.

The projected-straight-line calculating section 61 is to calculate astraight line passing a paper color and another representative color.This straight line is assumed as a projected straight line. FIG. 12Bshows straight lines that pass through a paper color denoted by a whitecircle and other three representative colors than that.

The replacing section 62 determines a distance between each of theprojected straight lines calculated at the projected-straight-linecalculating section 61 and the pixel-based color of input image toreplace. FIG. 12C shows vertical lines of from the pixel-based color toreplace onto the projected straight lines. The vertical line has alength that provides a distance of from the pixel-based color to replaceto the projected straight line. Selected is a representative color,other than the paper color, corresponding to the shortest projectedstraight line, to thereby replace the pixel-based color to replace intothe selected representative color. Incidentally, as for the color whosedistance is in a predetermined range from the paper color, replacementis into the paper color. In this manner, the pixel-based colors of theinput image are each replaced into any of the representative colorsincluding the paper color. This provides an output image whose colors inuse are limited to the representative colors.

Incidentally, how to select a representative color is not limited to theforegoing method. For example, such a method may be used that an averageis determined over the colors surrounding the pixel to replace on theinput image, to select a representative color closest in distance to aline segment including a differential vector between the average colorand the pixel to replace in the color space or to select arepresentative color most approximate in color to the pixel to replacewhen the differential vector gas a length equal to or smaller than alength. Naturally, other various known method may be used.

FIG. 13 is a block diagram showing a second exemplary embodiment of theinvention while FIGS. 14A and 14B are concept figures of a filtercoefficient for the filtering section 15. In the figures, the similarelement to that of FIG. 1 is attached with the identical reference, toomit the explanation thereof. 15 is a filtering section. The secondexemplary embodiment shows a configuration that the filtering section 15is provided between a histogram producing section 11 and a peak-colorextracting section 12.

The filtering section 15 performs a three-dimensionally filteringprocess for the histogram with a coefficient of the histogramdistribution provided greater as nearer to the gravity center of thedistribution, say, as shown in FIG. 14A, for the histogram produced atthe histogram producing section. For example, where the FIG. 14A agravity center of a histogram distribution exists in a position45-degree lower right relative to a subject pixel (central pixel in thefiltering region, in this example), it is possible to perform afiltering process by using such a coefficient matrix as shown in aconcrete example of FIG. 14B. Although showing herein is merely in a twodimensional for the sake of illustration, filtering is actually with useof a three-dimensional coefficient matrix in accordance with athree-dimensional positional relationship between the gravity center ofthe distribution and the subject pixel.

The three-dimensionally filtering allows such a region to readily remainthat frequency decreases at an end of distribution on a color space andin a direction going away from the center-of-gravity. Due to this, whenextracting a peak color at the next peak-color extracting section 12, itcan be made easy to extract a peak color existing at the distributionend that in nature has been highly possibly existed on the original.Therefore, the color featured in the input image can be extracted as apeak color and selected as a representative color.

The three-dimensional histogram, filtered in a three-dimensional form atthe filtering section 15, is delivered to the peak-color extractingsection where a peak color can be extracted out of the three-dimensionalhistogram. The other configuration is similar to that of the firstexemplary embodiment and hence is omitted to explain.

FIG. 15 is an explanatory figure of an example of a computer program, astorage medium storing such a computer program and a computer, whereinthe image processing apparatus of the invention is realized in itsfunction by means of a computer program. In FIG. 15, 71 is a program, 72is a computer, 81 is a magnetic optical disk, 82 is an optical disk, 83is a magnetic disk, 84 is a memory, 91 is a CPU, 92 is an internalmemory, 93 is a reader, 94 is a hard disk, 95, 96 are interfaces, and 97is a communicating section.

The image processing apparatus in the invention, explained in theexemplary embodiments and the modifications thereof, can be realized ina part or the entire of the functions of various sections, by means of aprogram 71 to be executed on a computer. In such a case, the program 71and the data for the program to use, etc. can be stored on a storagemedium to be read out by a computer. The storage medium is somethingthat can cause a change of energy, e.g. in magnet, light or electricity,at the reader 93 provided in the computer hardware resource inaccordance with a description form of a program, and transfer thedescription content of the program to the reader 93. For example, it maybe a magnetic optical disk 81, an optical disk 82 (including a CD, a DVDand the like), a magnetic disk 83, a memory (including an IC card, amemory card and the like), or the like. Naturally, the storage mediumsare not limited to those transportable in type.

By storing the program 71 in such a storage medium and placing thestorage medium in a reader 93 or interface 95 of the computer 72, theprogram 71 can be read out on the computer and stored in the internalmemory 92 or hard disk 94. By executing the program 71 by means of theCPU 91, realized is the function of the image processing apparatus ofthe invention. Otherwise, the function of the image processing apparatusof the invention may be realized as in the following. Namely, bytransferring the program 71 onto the computer 72 through a network, thecomputer 72 can receive the program 71 at its communicating section 97and store it in the internal memory 92 or hard disk 94 thereby executingthe program 71 by means of the CPU 91. Incidentally, the computer may beconnected with other various devices through the interface 96, e.g. adisplay for displaying information and an input device for the user toinput information.

Naturally, part of the functions can be structured by hardware or all ofthose may be structured by hardware. Otherwise, it is also possible toconfigure a program including the invention together with otherstructure. For example, configuration is possible as one programtogether with a control program in an apparatus including an imagereading device and image forming device, e.g. copier, in a manner toprocess for color limitation as to the image read out by the imagereading device. Naturally, in the application for other purpose,integration is possible with the program for the relevant use.Incidentally, it is needless to say that the computer 72 may be arrangedhaving any or both of the image reading device and the image formingdevice or other configurations.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theexemplary embodiments were chosen and described in order to best explainthe principles of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

1. An image processing apparatus comprising: a histogram producingdevice configured to produce a histogram in a color space from an inputimage; a peak-color extracting device configured to extract a pluralityof peak colors having a local maximum frequency of appearance of a colorvalue in the histogram; a representative-color selecting deviceconfigured to determine whether to unify the extracted plurality of peakcolors, based on a feature amount of the extracted plurality of peakcolors, and that selects a peak color as a representative color amongthe extracted plurality of peak colors when the extracted plurality ofpeak colors are unified, the feature amount comprising a lightnessdifference among the extracted plurality of peak colors and directionsof vectors in the color space, the vectors each showing a shortestdistance from one of the extracted plurality of peak colors to a linesegment connecting between a reference color and a dark color; and areplacing device configured to replace a color of each pixel in theinput image by the representative color.
 2. The image processingapparatus according to claim 1, wherein the representative-colorselecting device selects the peak color as the representative colorbased on at least one of a color saturation, a hue, a lightness, afrequency and a differential value of frequency, of the peak color. 3.The image processing apparatus according to claim 1, wherein therepresentative-color selecting device additionally extracts a peak colorexisting close to the line segment connecting between the referencecolor and the dark color or between the representative color and aprojection point of the representative color onto the line segment. 4.The image processing apparatus according to claim 1, further comprisinga filtering device configured to creates a three-dimensionally filteringprocess for the histogram so that a coefficient of a distribution of thehistogram becomes greater as nearer to a gravity center of thedistribution, wherein the peak-color extracting device extracts the peakcolor from the histogram subjected to the three-dimensionally filteringprocess.
 5. The image processing apparatus according to claim 1, whereinthe representative-color selecting device determines whether a peakcolor having a lowest lightness in the plurality of peak colors can beregarded as black, and wherein when the peak color having the lowestlightness is regarded as black, the representative-color selectingdevice sets the peak color as the dark color, and when the peak colorhaving the lowest lightness is not regarded as black, therepresentative-color selecting device sets a black color as the darkcolor.
 6. The image processing apparatus according to claim 1, whereinthe reference color is a color of a paper on which the input image isformed.
 7. A method for processing an image, comprising: producing ahistogram in a color space from an input image; extracting a pluralityof peak colors having a local maximum frequency in the histogram;determining whether to unify the extracted plurality of peak colors,based on a feature amount of the extracted plurality of peak colors, thefeature amount comprising a lightness difference among the extractedplurality of peak colors and directions of vectors in the color space,the vectors each showing a shortest distance from one of the pluralityof peak colors to a line segment connecting between a reference colorand a dark color; selecting a peak color as a representative color amongthe extracted plurality of peak colors when the extracted plurality ofpeak colors are unified; and replacing a color of each pixel in theinput image by the representative color.
 8. A non-transitory computerreadable medium storing a program causing a computer to execute aprocess for an image in an image processing apparatus, the processcomprising: producing a histogram in a color space from an input image;extracting a plurality of peak colors having a local maximum frequencyin the histogram; determining whether to unify the extracted pluralityof peak colors, based on a feature amount of the extracted plurality ofpeak colors, the feature amount comprising a lightness difference amongthe extracted plurality of peak colors and directions of vectors in thecolor space, the vectors each showing a shortest distance from one ofthe plurality of peak colors to a line segment connecting between areference color and a dark color; selecting a peak color as arepresentative color among the extracted plurality of peak colors whenthe extracted plurality of peak colors are unified; and replacing acolor of each pixel in the input image by the representative color.